FODAVA-Partner: Visualizing Audio for Anomaly Detection

“Most people who handle money a lot (i.e. cashiers) can identify a lower-quality fake bill instantly just by touching it” [10]. Data analysts are like cashiers: a trained data analyst can detect anomalies “at a glance” when data is appropriately transformed. This is the goal of data visualization. This proposal addresses the type of audio anomalies that human data analysts hear instantly: angry shouting, trucks at midnight on a residential street, gunshots. The human ear detects anomalies of this type rapidly and with high accuracy. For example, rifle magazine insertion clicks are detected with 100% accuracy at 0 dB SNR in white noise, babble, or jungle noise [1]. Unfortunately, a data analyst can listen to only one sound at a time. Visualization shows the analyst many sounds at once, possibly allowing him or her to detect an anomaly several orders of magnitude faster than “real time.” This proposal aims to to render large audio data sets, comprising thousands of microphones or thousands of minutes, in the form of interactive graphics that reveal important anomalies at a glance. Precedents for such graphical rendering are familiar to audio professionals. A simple amplitudeversus-time graph reveals silences for a speech transcriber to skip past; a spectrogram reveals details of birdsong to an ornithologist. But many audio anomalies are not so easy to display: angry shouts versus enthusiastically shouted greetings; the clatter of overturned tables rather than mere dishwashing; spoken Thai in Berlin, or German in Bangkok. All these cases can submit to automatic anomaly detection, using probabilistic models of longand short-term spectral features of normal activity. Unfortunately, the state of the art in automatic audio event detection is not very accurate [112]. We propose to represent audio anomaly to the analyst using a type of overcomplete lossless encoding: automatic anomaly salience scores will be displayed together with raw audio features, allowing a human analyst to drill down at any point in the data in order to resolve discrepancies in the visible display. The goal of this proposal is to present to analysts a coherent visual summary of both probabilistic and raw spectral information. The measurable outcome of this research will be the speed with which analysts find audio anomalies that have been planted, by the experimenter, in a very large dataset. A successful research outcome will be a visual summary that lets the analyst detect most anomalies immediately (about 10,000× faster than real time), and all anomalies after brief interactive exploration (about 1,000× faster than real time). In short, the goal of this proposal is to transform, model, and reduce data for efficient effective visualization and analytic reasoning: • A simple time series is transformed into audio features and probabilistic model-based features, vastly reducing the quantity of data presented at one time to the analyst (fewer observations). • Multiple techniques of dimensionality reduction condense the breadth of the data presented to the analyst (fewer variables). • Computationally inexpensive multiscale caching of all layers, from summary variables down to the raw source data, supports interactive investigation of hypotheses at different spatial and temporal scales. • The interactive visualizations are efficient: caching increases the analyst’s decision rate. • The visualizations are effective: the decisions have a measurably low error rate.